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Next-generation assessments and their 
implications for K-12 Education 


To illuminate the possibilities for next-generation assessments in K—12 schools , this case study profiles 
the Cisco Networking Academy ; which creates comprehensive online training curriculum to teach 
networking skills. Since 1997, the Cisco Networking Academy has served more than five million high 
school and college students and now delivers approximately one million online assessments per month 
in a variety of formats. Its advanced and highly integrated assessment system offers lessons for K-12 
technology and assessment. 


INTRODUCTION 

In 1996, Cisco Systems, Inc., an American multinational corporation headquartered in San Jose, 
Calif., that designs, manufactures, and sells networking equipment, discovered that although 
the computer networking industry was booming, there was a shortage of qualified candidates 
to design, build, manage, and secure computer networks. In response, the company integrated 
backward and founded the Cisco Networking Academy (NetAcad), a comprehensive online 
training curriculum offered to third-party education institutions to help high school and college 
students acquire the fundamental skills needed to design, build, and troubleshoot computer 
networks. NetAcad employs a system of assessments to help drive consistent learning across its 
training sites around the world. 

NetAcad launched in October 1997 at 64 educational institutions in seven U.S. states. Since 
its launch, the program has expanded to more than 9,000 high schools, colleges, technical 
schools, and community organizations in 50 U.S. states and 170 countries with its curriculum 
taught in 16 languages. More than five million students have participated in NetAcad courses 
and collectively taken 143.2 million online assessments. 

Cisco Systems credits its strong public-private partnership model for NetAcad s rapid growth. 
From the outset, the company envisioned a distributed delivery structure in which it would reach 
more students by partnering with existing educational institutions. As George Ward, former 
Cisco Systems engineer and founder of NetAcad, said, “Cisco [Systems] knows networking, 
teachers know education. By partnering, students win.” Cisco Systems provides NetAcad online 
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content, lab materials, and instructor training for free to institutional partners, which in turn 
provide classroom space, computer lab equipment, and qualified instructors. 1 

How NetAcad works 

Networking proficiency requires an array of hardware and software skills. The initial version 
of NetAcad offered a four-course sequence that provided students with the practical systems 
networking skills needed to obtain the Cisco Certified Network Associate (CCNA) Certification, 
a certification for entry-level networking jobs that an independent group separate from NetAcad 
administers. 2,3 Today, NetAcad s instructional model combines face-to-face facilitation by 
instructors; in-person, hands-on labs; and online learning. Cisco Systems provides participating 
educational institutions with a comprehensive package of instructional tools and curriculum, 
including hands-on lab materials; online practice activities; a simulation tool; a visualization and 
assessment engine called Cisco Packet Tracer; and educational games. The program also offers 
participating educational institutions discounted equipment bundles and a range of support 
services, including a global 2 4/7 NetAcad help desk. 

Cisco NetSpace 

NetAcad delivers its online curriculum and assessments through a cloud-based learning 
management system (LMS) called Cisco NetSpace. On Cisco NetSpace, students can navigate 
through online course material, complete assignments and assessments, and communicate with 
their instructors. The portal is available at all times so that students can complete activities at 
their convenience. 

On the back end, instructors use Cisco NetSpace to manage their courses. The site also 
serves as a management system for NetAcad administrators and has the functionality to simplify 
common administrative tasks such as enrollment tracking, scheduling, and managing equipment 
inventory. 

Because the system is cloud-based, it also supports the needs of curriculum and assessment 
developers at Cisco Systems. As developers change or create curriculum or assessment items, they 
can distribute these materials quickly and cost effectively across NetAcad’s 9,000 sites, which 
ensures consistency across the academies. 

Cisco Systems designed the NetAcad curriculum and instructional model to ensure that 
students develop a range of systems networking competencies, while accounting for different 
teaching and learning styles across the academies worldwide. With students in 170 countries, 
NetAcad seeks to be both “globally consistent and locally relevant.” 4 Even though the content and 
tools are standardized across sites, NetAcad recognizes the need for instructors to add context and 
additional information appropriate to their local economies. As such, the NetAcad curriculum is 
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designed to be instructor-facilitated — that is, instructors create classes in Cisco NetSpace, enroll 
students, and provide instruction. NetAcad designs tools for its instructors to manage each course 
and assign the online content and activities that align with their in-person sessions. Instructors 
can also use Cisco NetSpace tools to supplement the curriculum with activities and assessments 
that they build themselves. 

ASSESSMENTS 

Besides its impressive scale and distributed model, what sets Cisco Systems 5 network training 
curriculum apart is its emphasis on online assessment tools. As Cisco Systems developed 
NetAcad, it sought to rethink what it viewed as the traditional, flawed assessment paradigm. In 
many educational settings, assessments are used solely for accountability, grading, or promotion. 
Instead, NetAcad designers wanted assessments to provide instructors with feedback on student 
learning that they could use to inform instruction in real-time and provide students with up-to- 
date information on their progress and understanding of the material. 

NetAcad designers reframed the use of assessments as primarily for learning (formative) 
rather than evaluation (summative). Summative assessments allow instructors to evaluate student 
learning and award course credit; they remain critical to the goal of ensuring that NetAcad course 
completion and the subsequent certification is meaningful beyond the walls of the classroom. 
Formative assessments, on the other hand, help students, instructors, and administrators collect, 
analyze, and use information about student learning progressions to continuously drive and 
improve learning. To shift to primarily formative assessment, Cisco Systems built a broader and 
deeper portfolio of assessments, designed new technologies to assess how students grapple with 
content in real-time, and honed a process for creating consistently high-quality assessments. 

Today, NetAcad delivers more than one million assessments to roughly one million students 
each month. The following sections describe the new methods of assessment that Cisco Systems 
pursued in its expanded assessment portfolio; the design principles and data-driven cycles of 
improvement that guided this process; and the costs associated with NetAcad. 

Assessment portfolio 

Since launching NetAcad, Cisco Systems has developed a wide range of assessments that are flexible 
enough to meet the diverse needs of its students and instructors. Instructors have full discretion 
over how they use the assessments, including which assessments to deliver and whether to use 
a given assessment for formative or summative purposes depending on the needs of the course. 
Building an assessment portfolio was not a small task: computer-networking skills are fluid and 
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complex and thus can be challenging to assess. NetAcad students must not only learn the necessary 
content knowledge, but also the technical skills to design and construct physical networks, such 
as the organization and layout of hardware and cables, as well as business and problem solving 
skills that are more conceptual in nature. NetAcads assessments vary in complexity, length, 
and type. The assessments use constrained response (multiple choice, matching/drag and drop, 
fill-in-the-blank) to evaluate a learners content knowledge and skills; rubric-graded, in-person 
hands-on exams to assess manual skills; and simulation-based performance exams to evaluate 
more advanced problem solving skills. Table 1 summarizes the current assessment portfolio. 

NetAcads formative assessments are primarily intended to gauge progress, not evaluate 
performance. Much like quizzes in traditional courses, students take NetAcad quizzes that are 
embedded in the online curriculum after completing specific sections of content. Unlike quizzes 
in traditional courses, however, NetAcad created these quizzes for student self-assessment; they 


Table 1. NetAcad’s assessment portfolio (Source: Cisco Networking Academy) 



* Performance of networking skills is based on understanding. 


4 


CLAYTON CHRISTENSEN INSTITUTE 









are optional and not registered in an online grade book (although the instructor can change 
the attributes of the quiz to meet her classroom needs). Additional learning activities — such as 
short questions that check for understanding, modeling activities, reflections questions, and mini 
simulation exercises — are also interspersed throughout NetAcad’s online curriculum. 

On the back end, the NetAcad assessment system requires a large amount of processing power 
and sophisticated infrastructure to operate smoothly. As students complete summative and 
formative assessments in educational institutions across the world, data is immediately collected 
and then transferred back through the cloud. Students receive their scores within minutes. The 
grades also go directly to a database that instructors can access through the online grade book. 
NetAcad’s goal behind these short quizzes and activities is to make assessment as seamless as 
possible by tracking student performance during the learning process rather than after the fact. 

Other assessments may be used as either formative or summative. For example, some 
instructors use end-of-chapter tests as a formative exercise in the form of homework, whereas 
others use them as part of summative student evaluation as part of a student’s final grade. At the 
end of a course, NetAcad final knowledge exams provide summative assessments on the entire 
course content. Both end-of-chapter tests and final knowledge exams are taken online and scored 
automatically. In addition, students are required to complete a hands-on final exam under the 
observation of a supervisor. 

Performance assessments and simulations 

Although online assessment has always been a cornerstone of NetAcad courses, the assessments 
have evolved significantly since the programs inception. NetAcad’s initial model used a simple, 
fixed-response assessment system — single- answer, multiple-choice exams that tested content 
knowledge at various points in the course including a final exam at the conclusion. These online 
tests were complemented with an in-person, hands-on skills exam. As described above, some 
assessments were used formatively and others provided end-of-course summative evaluations. 
As the program developed, however, Cisco Systems sought to go beyond fixed response items to 
create performance-based tasks that could be scored automatically. Although scoring complex 
simulations can be challenging, these assessments present the potential for far more useful and 
extensive data. 

To assess and gather multiple forms of student data, Cisco Systems developed Cisco Packet 
Tracer 5 , a network simulation program that allows students to experiment with network behavior 
and ask “what if” questions. Cisco Packet Tracer is locally installed software that connects with 
the NetAcad cloud . 6 The technology offers a sophisticated simulation environment that presents 
a wide range of complex scenarios to students. In the Cisco Packet Tracer “micro world,” students 
can use numerous virtual devices to create and troubleshoot an infinite number of model 
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Figure 1. Example of a Cisco Packet Tracer activity 
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networks. Cisco Packet Tracer 
provides a graphical interface 
that allows students to design 
and build networks using simple 
drag-and-drop functions so that 
students can visualize processes 
that would otherwise be hidden 
in physical systems. 

A Cisco Packet Tracer activity 
that might be completed in a 
few hours* is shown in Figure 
1. Figure 2 shows the “answer 
network” tree, which illustrates 
some of the aspects of the 
network model being assessed. 
For the network topology shown 
in Figure /, there are over 2,000 
components that can be assessed. 
Instructors can use Cisco Packet 
Tracer for demonstrations, 
homework, and assessments. 

In 2010, Cisco Systems 
introduced the Cisco Packet 
Tracer Skills Assessments, which 
builds on the Cisco Packet Tracer 
simulation program to include 
standardized, simulation-based 
assessments. These assessments 
generally begin with either a network that is missing devices and has incorrect configurations that 
students must fix or an entirely empty starting point from which students must build a network 
from scratch. Unlike the open-ended Cisco Packet Tracer simulations, these assessments ask 


Figure 2. Example of the “answer network” tree for a Cisco Packet 
Tracer activity 


Time Elapsed: 00:00:36 


* The time frame of the learning activities varies depending on what the student is doing as well as on the student’s skills-level 
and fluency. Some learning activities — such as simple verification, configuration, or modeling tasks — could take a few minutes, 
whereas others — such as end-of-course skills assessments in the more advanced courses — could take a few hours. 
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students to achieve a particular 
outcome, sometimes while 
operating with constraints on 
the ability to use certain features 
and tools. 

Cisco Packet Tracer Skills 
Assessments are powerful 
formative assessment tools, as 
they provide students and 
teachers with a clear sense of how 
students are moving through 
material and solving problems of 
complex design, configuration, 
and troubleshooting tasks. 

Figures 3 and 4 show the types 
of feedback that instructors and 
students can receive from Cisco 
Packet Tracer Skills Assessments. 

All of a students activities or 
“moves” in the system can be 
recorded in a time-stamped 
log to provide data on how the 
student built or manipulated 
a network. Both students and 
instructors can obtain and 
analyze these log files. The system also compares students’ completed networks to an “answer 
network” that identifies specific, desirable features in the environment. The system awards credit 
for ranges of correct answers, rather than simply right or wrong responses. The system then 
provides each student with rich feedback outlining the specific competencies the assessment 
tested and how he or she performed on them. 

Instructors not only can select from the NetAcad-provided assessments but also design their 
own assessments and activities through both a multiple-choice quiz generation tool as well as the 
more sophisticated Cisco Packet Tracer Wizard, to design activities and assessments within the 
simulation system. The Cisco Packet Tracer Activity Wizard allows the instructor to control the 
instructions presented to the student, timing of the activity, initial network (from completely 
empty network to an almost complete one), and final network states (with thousands of student 


Figure 3. Example of feedback instructors and students can receive 
from Packet Tracer Skills Assessments 



Figure 4. Example of feedback instructors and students can receive 
from Packet Tracer Skills Assessments 
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work product features that can 
be scored). Instructors can use 
these activities to supplement the 
standard NetAcad curriculum 
and assessments. Figure 5 shows 
how instructors, by simply 
placing check marks next to 
specific model features, can 
specify which aspects of a model 
are assessed. 

Cisco Systems found that 
simulation software like Cisco 
Packet Tracer allows students to 
experiment with how networks 
operate, which enhances their 
hands-on work in the physical labs. Cisco Packet Tracer may also mitigate resource constraints 
among some NetAcad partner educational institutions. Although NetAcad requires that partner 
institutions use physical equipment to train students, schools may have fewer laboratory tools 
than students. Students often have to wait to use equipment and are constrained to practicing 
when they are on site. Simulated performance exams can therefore greatly expand access for 
students who are working remotely or for resource-strapped partners with crowded labs. 

Assessment design and development 

Evidence-centered design 

As NetAcad embraced the task of building a rich portfolio of formative and summative 
assessments, the assessment team sought to identify a methodology that could tie together and 
standardize all its assessments, as well as ensure that the assessments were of consistently high 
quality. The method also needed to be sufficiently flexible and abstract to apply to all types of 
assessments, including as the assessments grew in technological sophistication over time. 

NetAcad s assessment team adopted an approach called evidence-centered assessment design 
(ECD) to develop the assessments. Researchers at the Educational Testing Service (ETS) — 
the author of the SAT, GRE, and other standardized entrance exams — introduced the ECD 
framework. ECD is organized around answering the question, “how do we know what students 
know?” The framework focuses on examiners presenting the right activities to find evidence 
of student knowledge and skills in a way that supports fully the underlying purpose of a given 


Figure 5. Example of how instructors can specify which aspects of a 
model are assessed 



8 


CLAYTON CHRISTENSEN INSTITUTE 


assessment. It allows assessment designers to start by asking what skills and knowledge should be 
assessed, then understanding what behavior will reveal them, and finally developing situations 
that elicit that behavior. 

Although ECD is not prescriptive, it does lead to specific ways of iterating to design, define, 
develop, analyze, and refine the assessment. The flexibility of the framework makes it particularly 
relevant for a more fluid and activity-based domain such as networking. As such, ECD has 
been critical to building the complex assessment tasks, in particular the Cisco Packet Tracer 
simulations. 

Designer capacity 

To populate its wide portfolio of assessments using ECD, NetAcad employs instructors from 
across the world as consultants to create and develop the assessments. The consultants receive 
extensive training from NetAcad in the design of high-quality assessments, including the 
conceptual understanding of ECD. Consultants develop items virtually through a cloud-based, 
assessment-authoring software — available in 1 9 languages — that facilitates broader participation 
by a global base of instructors. The individual who prepares each assessment activity documents 
the particular curricular goal that the question should address and drafts the initial assessment 
item. Then each item is reviewed in a multi-stage process to assess quality. The online system 
allows developers to work on assessment design together, manage workflow, and offer feedback 
and review in an online community. This collaborative process ensures that there are multiple 
eyes on every question and facilitates ongoing debate, which increases the overall quality of 
assessments. Small focus groups of instructors also test items before Cisco Systems launches the 
items to the entire academy. 

Data and feedback 

Two lead NetAcad researchers, John Behrens and Kristen DiCerbo, have described the shift from 
simple assessments to simulation tasks as moving to a “digital ocean,” where assessment data, rather 
than being scarce and difficult to obtain, is ubiquitous. NetAcad s assessment system produces 
an enormous amount of data about the students, instructors, and curriculum itself. NetAcad 
uses this data to make improvements at the student, classroom, and academy level. Because the 
majority of assessment takes place online, student performance data is available instantaneously. 
All NetAcad authored assessments are graded automatically and scores can be included in the 
online grade book, depending on the assessment’s purpose. Instructors can review detailed scoring 
and performance data through Cisco NetSpace. They can view the data at multiple levels — 
from aggregated class-level information to detailed item-level performance by a student that 
shows what skill the item was intended to test and how the student performed on it. The system 
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even automatically scores and 
provides feedback on Cisco 
Packet Tracer Skills Assessments, 
including final network and log 
files. Figures 6 and 7 show an 
instructors view of a students 
progress in Cisco NetSpace. 

Assessment consistency 
across the entire network also 
facilitates greater data use 
among instructors. Assessment 
sequences across courses are 
unscarefully planned so that 
practice opportunities, skills 
integration challenges, and 
chapter exams all build toward 
preparing students for the end- 
of-course online multiple-choice, 
simulation-based performance, 
and hands-on performance 
exams. Instructors can compare 
their students’ progress with 
similar academy classes around 
the world, which gives them a better sense of how their students should be performing along the way. 

Student performance data provides a means to identify strengths and weaknesses in NetAcad s program as 
a whole. Cisco Systems assessment team uses standard psychometric statistical processes to identify in which 
content areas students are struggling as well as which assessment questions are confusing or can be improved. 
NetAcad makes the results of these analyses available to developers and consulting instructors to inform future 
assessment development. 

NetAcad also uses data to manage quality at the site level. Given the decentralized nature of the academy — 
thousands of sites across hundreds of countries — it is difficult to monitor all sites to ensure that high-quality 
standards are maintained across the network. To supplement more traditional support, the central Academy 
Quality Assurance Team uses the information collected from online assessments to scan for site-specific issues. 
This group also monitors student performance on a class level to identify trends in the data, such as consistently 
low scores for students at a given academy site, which may indicate institutionally weak instruction. Regional 
support teams also receive this data, which allows them to target their support to individual sites. 7 


Figure 7. Example of an instructor’s view of course analytics 
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Figure 6. Example of an instructor’s view of a student performance 
summary 
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In addition to the quantitative data analysis, instructors can submit feedback to the assessment 
development team. For example, if a particular question is confusing or inaccurate, instructors 
can flag it for review in the online system. If the issue is found to be valid, then the assessment 
development team updates the item or exam and the instructor is informed. Designers can use 
this data to adapt the program with greater accuracy and speed than if they had to wait for the 
completion of an annual assessment cycle, and they can take advantage of the on-the-ground 
expertise of the academy instructors. Also, because exams are launched online, Cisco Systems can 
seamlessly pilot the introduction of new questions to a small group and also make replacements 
once improvements are identified. Cisco Systems also uses student perception data to improve 
the academy. At the end of each course, students use the assessment system to complete a survey 
that provides feedback on the course and the instructor. This data influences refinements to the 
course as a whole as well as provides feedback for each instructor and site. 

Assessment data provides useful feedback for students as well. Students can view the results 
of their assessments online and use the results to understand better what material they need 
to review. Furthermore, students receive a personalized-learning report after taking a chapter 
and practice final exam or a Cisco Packet Tracer Skills Assessment, which provides detailed 
information on performance and refers the student back to the curriculum. 


COSTS 

As of 20 1 1 , Cisco Systems had invested $400 million in NetAcad since the program’s inception 
in 1997. This includes the cost of curriculum and assessment development, technology, 
marketing, and general operations, as well as in-kind contributions from Cisco Systems of more 
than $212 million. Although the per-course costs were hundreds of thousands of dollars in initial 
investment, recent engineering and process improvements have substantially decreased this cost 
and also established the technical basis for substantial reuse. 

Although these initial investments may seem high, Cisco Systems quality assurance measures 
save the company tremendous money in translation and maintenance. Even though the 
development and automatic scoring of assessments require significant upfront programming 
costs, they scale well — because the assessments are standardized across all sites, the marginal 
cost to deliver and score assessments for a new site is minimal. This allows Cisco Systems to 
invest the majority of its funds in developing the best possible assessment content and ensuring 
that the technology, both the student-facing software and backend infrastructure, is robust and 
sophisticated enough to handle the desired system elements. When spread across NetAcads 
approximately 10,000 sites and million students, these investments yield a low cost per student 
and course. 
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The additional variable costs per new site are small and covered primarily by the sites. 
Nonprofit organizations that serve as local academies pay nothing for access to the curriculum 
and assessment — they only pay for their own networking equipment and for training and support 
costs. As part of its corporate social responsibility program, however, Cisco Systems subsidizes 
the costs of NetAcad to participating educational institutions. This makes it easier for more 
educational institutions, particularly those in developing nations, to participate in the program 
and serve increasing numbers of students. 

Because few, if any, educational programs can boast the scale of NetAcad, other organizations 
may have difficulty replicating the sophistication of the assessment program at a reasonable cost 
per site. But many key elements of the model — such as network-enablement using high-quality 
cloud technologies, centralized experience design and psychometrics, and local delegation of 
instructional decisions — are plausible for smaller sites to consider. 


7 TAKEAWAYS FOR K-12 EDUCATION 

Although the NetAcad system of learning and assessments is not perfect, it offers several lessons 
for improving learning and assessment in the K-12 education system. 

1. Integrate assessment with curriculum and instruction 

NetAcad demonstrates the importance of aligning assessment with curriculum and instruction. 
Assessment is not an afterthought, but is considered an integral part of the NetAcad curriculum. 
The individuals that work on each component work closely together to ensure that there are 
strong links and that assessments not only accurately reflect the content, but also fit with the 
instructional delivery. NetAcad also relies heavily on the input of instructors in the initial 
assessment design and ongoing refinement process. This ensures that assessments are tied to the 
realities of what is happening in the classroom. 

Because NetAcad provides all of the components of a comprehensive online curriculum, 
Cisco Systems is able to accomplish this tight integration between assessment and instruction; 
it develops the standards for what students should learn in each course, the content for the 
courses, and the associated assessments. In our K-12 education system, these components are 
developed separately and often do not align perfectly to reinforce one another. External vendors 
generally create both summative and formative assessment products outside of the context where 
the instruction actually happens. Although summative assessments may be successfully executed 
without being highly integrated with instruction, this is not true of formative assessments. When 
formative assessments are done well, they are embedded in the instruction to both reinforce 
learning and inform instruction. 
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2. Leverage an online delivery system 

NetAcad’s online-delivery system is the backbone of its assessment program. As opposed to 
traditional pencil-and-paper assessments, the online portal delivers assessments with greater ease 
and lower marginal costs. This saves time for instructors and encourages them to use the optional 
assessments in a more frequent, formative manner. In addition to the day-to-day delivery and 
scoring of exams, the online system supports the mission of continuous improvement, discussed 
more below, by allowing updates to assessments to be rolled out seamlessly. Such an approach 
would shorten the development cycle for K-12 assessments, which can lag updates in curricular 
content. 

Another benefit of NetAcad s online system is its instantaneous scoring and feedback. Because 
Cisco Systems sees assessment as primarily learning-based rather than evaluative, it seeks to 
deliver better data — particularly through the scoring of complex tasks through its simulation 
program — that can be used quickly. In the traditional K-12 classroom, teachers struggle to find 
the time to grade complex tasks. But with automatic scoring and the appropriate analytical tools, 
teachers could learn rapidly about what their students know and adjust instruction quickly. 

3. Deploy a cycle of continuous improvement 

NetAcad uses a continuous feedback loop to improve the quality of its assessments. Because 
Cisco Systems views assessments as key to the efficacy of the academy, it has invested heavily in 
bringing top talent and resources to assessment development — both to develop new, innovative 
types of assessments as well as to improve individual items. 

Assessment is also critical to improving the broader instructional program at NetAcad. At the 
classroom level, assessment provides the data for teachers to determine what concepts they must 
reteach and identify ways to support individual students. Instructors also use assessment data 
to gauge their practice by seeing how their students are performing relative to students at other 
sites. Academy administrators can also review the assessment data to monitor course quality and 
provide support to struggling teachers. In the K-12 space, this notion has become a political 
battleground because under new teacher evaluation regimes, the use of data has been perceived 
as punitive in nature. NetAcad, however, encourages transparency and uses assessment data for 
instructors as it does for students — to understand strengths and weaknesses and inform strategies 
for improvement moving forward rather than merely for accountability. 


Cisco Networking Academy 


13 


4. Develop a diverse assessment system 

NetAcad uses a diverse portfolio of assessments to achieve its instructional goals. The multiple 
types of assessment vary not only by content tested but also by question type, delivery mechanism, 
duration, and instructor versus student control. Each assessment is targeted at a particular learning 
need and serves a specific purpose. Using a range of assessments also allows for the possibility of 
multiple measures to evaluate student learning. The assessments form a system that can address 
different needs, but is still cohesive and aligned to clear learning outcomes. 

Most of the energy in the K-12 education sector has historically been focused on summative 
exams, in particular statewide standardized assessments to evaluate learning at the end of a 
school year. Formative assessments, however, are increasingly recognized as a necessary tool to 
continually improve learning. 

Ultimately, NetAcads diverse assessment system and its flexibility should be an entree to 
move toward a competency-based learning system in which time is variable but learning is 
constant. In this system, assessments could simultaneously and interchangeably be used both 
for learning — formative — as well as for determining mastery and progression — high stakes or 
summative. These “moderating” assessments could help break false trade-offs in assessment that 
exist because of todays current system that holds time as a constant and learning as a variable. 

5. Invest in new ways to assess complex tasks 

NetAcads assessment portfolio also includes tests that are both objective and subjective in nature. 
Subjective assessments allow for a wider range of answers, which in turn can account for more 
complex tasks — such as writing, critical reasoning, and other 21st-century learning skills — and 
can adjust for differences among students. Most standardized assessments test simple content 
knowledge and factual recall but have trouble assessing complex skills. 

NetAcads Cisco Packet Tracer simulations and games provide better visualization of 
the networks than physical equipment because they can show the logic that underlies the 
actual equipment. And, unlike a physical network, the simulation offers instant support and 
troubleshooting to remedy issues even without the instructor present. Furthermore, the simulated 
world allows students to experience the large and complex networks that students may encounter 
in the real world. Cisco Packet Tracer can encourage experimentation and discovery, which allows 
students to test their understanding in a safe learning environment with endless possibilities for 
scenarios, which can make for even more robust learning. 

Because these elaborate simulations can be scored automatically, Cisco Systems simulations 
lend hope to the K— 12 education system that we can measure things that were previously too 
expensive or challenging to evaluate. Advancements in technology are making it possible to 
deliver and score such exams at reasonable costs. 8 


14 


CLAYTON CHRISTENSEN INSTITUTE 


6. Use a rigorous but flexible design process 

NetAcad uses a design process — evidence-centered design (ECD) — to enable assessment 
developers to create accurate assessments of all types. ECD is a framework, not a prescriptive 
model, which is flexible enough to be useful even as the program changes and new assessment 
formats are developed. This is particularly relevant to the K— 12 education system as new school 
models have the potential to change the nature of education. A flexible design process ensures that 
assessments can keep up with educational innovation. Notably, the two major Common Core 
assessment consortia — the Smarter Balanced Assessment Consortium and the Partnership for 
Assessment of Readiness for College and Careers (PARCC) — are using ECD as the framework 
to design their assessments. 

7. Balance standardization with local customization 

NetAcad’s largely centralized development of curriculum and assessments offers a number of 
advantages. First, it maintains the quality of courses by providing standard measures for student 
and teacher performance. Second, it allows individual academies and instructors to measure 
their progress against other sites. Third, it reduces costs because individual instructors do not 
have to develop their own assessments. Historically in the U.S. K— 12 education system, such 
standardization would have been difficult because each state had different academic standards. 
With the emergence of Common Core in English language arts (ELA) and math, however, 
there is some momentum — although it is hardly universal — for common assessments. The K— 12 
education sector can look to NetAcad for how to facilitate local customization even within a 
common assessment system. NetAcad recommends but does not require any assessments, thereby 
allowing instructors to select which assessments are appropriate for their students. Instructors can 
also develop their own formative and interim assessments on top of the standard course exams. 
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CONCLUSION 


NetAcad leverages advanced technology not only to rethink content development and distribution, 
but also to rethink how to assess student learning and instructor and course efficacy. Using next- 
generation technologies to deliver continuously formative assessments represents a radical shift 
in learning and assessment. The K— 12 education space is witnessing growing enthusiasm for 
using technology to deliver and score assessments online. Indeed, many states participating in 
Common Core will opt for the associated assessments to be taken online. These efforts may have 
the potential to reduce costs to scoring summative assessments. But they will not confer the 
potential benefits of ongoing formative assessment if these online assessments simply replicate 
traditional pen-and-paper tests in an online format. Cisco Systems’ experience suggests that next- 
generation assessment at scale is possible — if not expensive and contingent on strong integration 
of content, assessment, and instructor training. 
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NOTES 

1 After a major course revision cycle that finished in November 20 1 3, NetAcad’s course offerings include Information Technology 
Essentials, Introduction to Networking, Routing and Switching Basics, Routing and Switching Essentials, Scaling Networks, and 
Connecting Networks. 

2 Cisco Systems has developed a rigorous, industry-standard system that confirms the ability to build, operate, and troubleshoot 
computer networks. There are five levels of certification representing increasing expertise and nine different content paths such as 
Routing and Switching, Design, Network Security, Voice, and Wireless. 

3 In order to ensure practical relevancy, Cisco Systems developed the content for each course by identifying the competencies 
required for each job in the evolving information and communications technology market. Given the continually changing nature 
of the industry, NetAcad updates its curriculum and course offerings every three to four years and makes smaller updates on an 
ongoing basis. 

4 “Cisco Networking Academy Program at a Glance,” Cisco Networking Academy, http://www.cisco.com/web/about/ 
citizenship/socialinvestments/docs/NetworkingAcademy_External.doc (accessed September 16, 2014). 

5 Cisco Packet Tracer Skills Assessments, along with the NetAcad online curriculum, is hosted on Cisco Systems’ learning 
management system (LMS), NetSpace, through its satellite system Virtuoso Delivery System. Cisco NetSpace offers extensive 
tools to manage assessments at the site level, including the ability for instructors to administer exams to any selection of students 
in their classes and the automation of grading and other administrative tasks. Virtuoso Delivery System has additional features to 
increase ease of assessment delivery, such as allowing the instructor to distribute students the same questions but in random order. 

6 When running locally, Cisco Packet Tracer can interact with the cloud-based Virtuoso Delivery System, a satellite system that 
delivers all non-quiz assessments. 

7 The size and scope of NetAcad necessitates a tried delivery model for instructor training and site support — train-the-trainers. 
Cisco Systems selects and prepares Cisco Academy Training Centers that provide training and support to regional academies 
that in turn provide support to local academies. Both regional and local academies are educational institutions that offer courses 
to students; regional academies are local academies that were promoted based on their performance and capacity to support 
additional academies. This model ensures that there is appropriate support for all academies, no matter how many sites, and also 
that training achieves a balance between global standardization and local customization. The instructors receive suggestions and 
some guidance from the Networking Academy Instructor material, but are directed to implement their instruction following their 
own local conditions, constraints, and resources. 

8 Hewlett-Packard’s recent competition to develop a technology to analyze and grade student-writing samples is an encouraging 
step in this direction. See “Hewlett Foundation Sponsors Prize to Improve Automated Scoring of Student Essays,” The William 
and Flora Hewlett Foundation, January 9, 2012, http://www.hewlett.org/newsroom/press-release/hewlett-foundation-sponsors- 
prize-improve-automated-scoring-student-essays (accessed September 16, 2014). 
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About the Institute 

The Clayton Christensen Institute for Disruptive Innovation is a nonprofit, nonpartisan think 
tank dedicated to improving the world through disruptive innovation. Founded on the theories 
of Harvard professor Clayton M. Christensen, the Institute offers a unique framework for 
understanding many of society’s most pressing problems. Its mission is ambitious but clear: work 
to shape and elevate the conversation surrounding these issues through rigorous research and 
public outreach. With an initial focus on education and health care, the Institute is redefining 
the way policymakers, community leaders, and innovators address the problems of our day by 
distilling and promoting the transformational power of disruptive innovation. 

Acknowledgements 

Thank you to the experts at Cisco Networking Academy for their contributions to this case study. 


About the author 



MEREDITH LIU is a visiting research fellow at the Clayton Christensen 
Institute. She has worked with a diverse range of organizations in the K— 12 
education space, including nonprofits, school districts, state education 
agencies, and charter management organizations. Most recently, she 
served as chief financial officer at Match Education, an innovative charter 
school network and teacher training program. She began her career as 
a consultant at Bain & Company. Meredith is currently leading a new 
initiative at Startup: Education. She holds a BA from Dartmouth College 
and an MBA from the Harvard Business School. 



Copyright © 2014 by Clayton Christensen Institute 
All rights reserved, 
www. christenseninstitute . org 


